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Gesture-based input device for a user interface of a computer 
Technical Application Area 

The present invention relates to a gesture-based Input device for a user 
interface of a computer and a new Interface technology enabling gesture- 
5 based navigation and interaction, searching, browsing and selection of web 
pages and multimedia content graphical user Interfaces (GUIs). 

State of the Art - Drawbacics of current technology 

10 The most commonly used technologies for gesture-based Interfaces Include 
data gloves, computer vision techniques and magnetic or Infrared sensors. 
Data gloves are expensive and also restrains users to a device; also, data 
gloves are not suitable for use in public spaces. Computer vision systems are 
sensitive to lighting conditions and have a comparably low update rate, though 

15 they can be quite accurate. But vision systems often force the user to adapt a 
new means of Interaction (e.g. head or eye movement). Infrared and magnetic 
sensors also require the user to wear a device. EFS-based navigation 
techniques require users to undergo tedious calibration procedures, 

20 Moreover, gesture-based touchless Input devices for user interfaces of 
computers are known using cameras for determining the position and gesture 
of e.g. a user's hand. Examples of these technologies are described in DE-A-42 
01 943, DE-A-199 18 072, DE-A-100 07 891, DE-A-100 56 291, US-B-6 252 
598, JP-A-09258945 and JP-A-10031551. 

25 

Moreover, from DE-A-43 12 672 a touchless cursor-control-like input device 
for computers is known which comprises proximity switches for controlling 
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cursor movement and other functions. Each proximity switch functions based 
on a capacitance measuriement. These switches cannot detect the position of 
for example a user's finger within their sensitive area but can merely detect 
that within the sensitive area a user's finger is located. 

5 

Solution provided by the New Development 

The Invention provides a gesture-based Input device for a user interface of a 
computer comprising 

10 - two pairs of antennas wherein the antennas are arranged foi- surrounding 
a two-dimensional sensor field corresponding to the user interface and 
providing different options or taslcs to be selected by a user, 
a platform for supporting a user, 
a DC power supply connected to the platform and 

15 - a circuitry connected to the antennas for determining, relative to each of 
the antennas, the position of that part of a user supported by the 
platform, e.g. a user's hand, being closest to the sensor field, 
wherein the position of the part of the user in each dimension of the 
sensor field is determined based on the relation of two voltage signals of 

20 the circuitry, respectively, each voltage signal indicating the distance 

between the part of the user and the respective antenna limiting the 
respective dimension of the sensor field, 

whereby the position within the sensor field closest to the part of the user 
is determined without any calibration of the sensor field to the user. 

25 

Individual embodiments of the Invention are referred to in the dependent 
claims. 

The system according to the Invention is an alternative to the computer mouse 
3 0 or the touchscreen, because It allows the user to command and control the 
cursor via plain pointing gestures. The system can be used with all standard 
monitors and especially (large) wall projections. The interface according to the 
invention does not require users to wear or hold dedicated devices; in addition 
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it does not need to be calibrated for different users. It is tlius easy to maintain 
and protect from damage. The system is thus aimed at Information systems In 
public spaces, such as information kiosks or multimedia display windows, 
which constantly have to cope with these Issues. 

5 

Basic Features of the Solution 

Using an array of electrodes, the system according to the Invention generates 
a weak electric field that reacts to the capacitance of the user. Body 

10 movement (e.g. by arm or hand) results in a change of the electric field, which 
in turn is measured and analysed by the system in order to measure the 
proximity to each of the electrodes; the exact position of the arm or hand can 
then be computed using standard approximation techniques. An algorithm 
allows it to use the system without prior calibration. Users can therefore 

15 control the cursor on the screen by simple pointing gestures. Objects can be 
selected by e.g. pointing at the same object for few seconds. 

The technology is largely built on two areas: the hardware developed Is based 
on highly sensitive electric field sensor (EFS) and an array of antennas. The 
20 software was designed and implemented to analyse the user interaction via 
the sensor array and estimate the arm position from it in order to enable 
browsing and selection. This includes an algorithm for adaptive relative 
reference which compensates for omitted calibration step. 

25 Improvements and Advantages over the State of the Art 

The system according to the Invention Is a non-contact Interface that doesn't 
require the user to calibrate the system or wear a device. He can Interact with 
the computer by pointing at the screen in a very natural and playful way. It 
30 provides good resolution and a high update rate (up to IkHz) for browsing web 
pages and multimedia applications. 
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The invention will be described in more detail herelnbelow referrlngto the 
drawing in which 

Fig. 1 shows an embodiment of the invention (referred to as Info-Jukebox)^ 
Fig. 2 shows a principle of the sensor, 

Fig. 3 shows a natural electromagnetic field around human body, 
10 Fig. 4 shows an enhanced human body electromagnetic field. 
Fig. 5 shows an electronic circuit model for the sensor, and 
Fig. 6 shows an antenna frame for the sensor. 
Example 



15 



The Info-Jukebox is a multimedia archive Installation for public spaces (see 
Fig. 1). It presents collections of multimedia information on a multi-screen (on 
20 multiple screens) which can be accessed via a gesture-based interface. ECCO- 
sensors translate simple pointing gestures into screen coordinates in order to 
control the cursor. The selected information (e.g. videos) is presented on the 
navigation space as well as on a large projection to ensure that the Info- 
Jukebox can be installed in large public spaces. 



25 



Technical Principle 



In order to ensure the basic functionality of the jukebox, two types of 
information are necessary: one is the current position of the cursor, and the 
3 0 other is selection of an item. Both parameters should be determined from the 
user's arm position. And indeed, one can do so by applying basic physics 
theory: It is well-known that capacitance C exists between two objects and C 
is proportional to 
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(1.1) 



where d is the distance between the two objects and K is the factor of the 
object characteristic and environment effect. 

5 

The capacitance C is a basic parameter in electronic science. By obtaining the 
value the distance between the two objects can be determined. Moreover, if 
two objects Al and A2 are placed at fixed positions (see Fig. 2) in a coordinate 
system, the position of the new object B can be determined by measuring the 
10 capacitance between the object pairs AlB and A2B. 

This is the basic principle underlying in the sensor of the present invention. B 
corresponds to the arm of the user. Al and A2. Once the distance is measured 
between Al and B and A2 and B, one can infer the position of the object B. By. 
15 means of suitable software routines, one can move the cursor on the screen in 
accordance with the movement of the arm. 

It Is widely known that the human body can radiate (or reflect) energy in the 
form of heat, infrared waves, electromagnetic power and others. Especially, 

2 0 the electromagnetic field is most closely related to the body, since it may be 
caused by the biochemical effects in the cells, friction of clothes, space electric 
field induction or coupled from power cable. This is why sometimes people 
may feel shocked by body electricity, especially when one takes off his/her 
wool sweater in dry winter day. Indeed, the human body is surrounded by an 

25 electromagnetic field more or less all the time, as can be seen in Fig. 3. 

Given such field, body gestures can be captured by suitable sensors and then 
translated into computer commands. One of the most common gestures in life 
Is finger or arm pointing gesture. This kind of gesture language has been 
30 widely adopted in human computer interfaces based, for example, on 
computer vision or others. Once it is possible to detect the change of the field 
cause by body movements, one can use it to control computers in some ways. 



e.g. to navigate a screen cursor or to emulate isolated buttons on the 
keyboard. 



Unfortunately, the natural electromagnetic field around the human body is 
extremely weak and difficult to deal with for human computer Interaction, 
whereas artificially created fields are easier to control. 

When the body is immersed in an electromagnetic field, it will become a 
medium or path for the field and extends to the surrounding environment as 
illustrated in Fig.4. In fact, the human body field or energy can be enhanced in 
this way to invisibly extend the measurable size of the body. 

In the method, the enhanced field is measured by a voltage coupled solution 
accompanied by a custom-designed frame of four or more antennas. 

The principle of the sensor may be expressed In the model of Fig. 5. 

When the user is enhanced by the artificial field, the field around him can be 
regarded as a signal source represented in the model as Ui. When the user 
moves his arm close to the antenna(s), the path between user and sensor 
antenna can be described as the equivalent capacitance C in the model, via 
which the signal passes to the front processing circuit and then to the 
computer. 

From Fig. 5, the output of the antenna Uocan be obtained as follows: 




R2 



(0.1) 



where |i/o| and \Ui] are the amplitude of the signal, 
Rl and R2 are the resistors used, and 
/ is the frequency of the signal 
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Since /is in the range kHz and C Is In pF unit, Rl « V-^^ • It follows that, 

{27fC) 

\U<^^27^R2\Ui\ (0.2) 

5 

So the measured amplitude of C/o contains the distance information of arm and 
antenna. 

For different users however, the electrical property is different. It is impossible 
10 to couple the same voltage strength Ui to everyone without careful 
calibration- In orderto overcome this shortcoming, an array of antennas as 
demonstrated in Fig. 6 is used. 

The prototype antenna frame is made from a 50 x 50 cm metal cannulation 
15 accounting to four antennas- Compared to the size of the hand, it can be 
regarded as the line and point problem for capacitance which, resulting in 
homogeneous reception characteristics for the antenna frame. 

The prototype frame consists of four antennas and each has a processing 
20 circuit as discussed above (cf. Fig. 5). If the signal from antenna a Is named^as 
\Uo\^ and the signal from the antenna c as \Uo\^ one can infer 

\Uo\ c 

H^\-^ = ^ (0.3) 

25 and are equivalent capacitance between the arm and the antennas a 

and c, respectively. 

Since capacitance is proportional to 

30 Coc— (0.4) 
d 

where d Is the distance between the two objects and K is the factor of the 
object characteristic constant and environment effect. Thus: 
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da 

d is the distance between the user's hand and antenna a, and likewise for 
to antenna c. From this equation, it can be seen that the value of H includes 
5 the relative distance information between the user's hand and the antenna. 
Furthermore, one can cancel out the uncontrollable effects to the environment 
Interference. 

If the last value of H\s stored as and the current one as H^, the. 
10 difference is — 

AH=:-H^-H^ (0.5) 

One now can compute A/f, and use this parameter to control the relative 
15 movement of the cursor. Thus, the position change is used to move the cursor 
and not the position itself which makes It work analogous to a mouse. 

For the selection control of the JukeBox, simple Boolean logic is used- It Is 
assumed that the user makes a selection when he drops down his arm. When 
2 0 the arm is raised, the signal strength of all four signals will increase, with the 
opposite effect when the arm is dropped. 
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CLAIMS 

1. Gesture-based input device for a user interface of a computer comprising 

two pairs of antennas wherein the antennas are arranged for 
surrounding a two-dimensional sensor field corresponding to the user 
interface and providing different options or tasks to be selected by a 
user, 

a platform for supporting a user, 

a DC power supply connected to the platform and 

a circuitry connected to the antennas for determining/ relative to each 
of the antennas, the position of that part of a user supported by the 
platform, e.g. a user's hand, being closest to the sensor field, 
wherein the position of the part of the user in each dimension of the 
sensor field is determined based on the relation of two voltage signals 
of the circuitry, respectively, each voltage signal indicating the 
distance between the part of the user and the respective antenna 
limiting the respective dimension of the sensor field, 
whereby the position within the sensor field closest to the part of the 
user is determined without any calibration of the sensor field to the 
user. 

2. Gesture-based input device according to claim 1, wherein movement of 
the part of the user within a plane parallel to the sensor field is 
determined based on the difference between a former position of the part 
of the user determined and the actual position of the part of the user 
determined. 

3. Gesture-based input device according to claim 2, wherein, when the 
determined position of the part of the user is left substantially unchanged 
for a predetermined period of time, this is interpreted as selecting an 
option or task offered to the user through the user interface represented 
by the sensor field. 
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Gesture-based input device according to any one of claims 1 to 3, 

wliereih the senosr field comprises a screen and a cursor moved and 

positioned according to the movement and position of the part of the 
user. 
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Abstract 

f;*.g»iirf»-based input devic e for a user interface of a computer 

The gesture-based input device for a user interface of a computer comprises 
two pairs of antennas wherein the antennas are arranged for surrounding a 
two-dimensional sensor field corresponding to the user interface and providing 
different options or tasks to be selected by a user, a platform for supporting a 
user, a DC power supply connected to the platform and a circuitry connected 
to the antennas for determining, relative to each of the antennas, the position 
of that part of a user supported by the platform, e.gl a user's' hand, being 
closest to the sensor field. The position of the part of the user In each 
dimension of the sensor field is determined based on the relation of two 
voltage signals of the circuitry, respectively, each voltage signal indicating the 
distance between the part of the user and the respective antenna limiting the 
respective dimension of the sensor field, whereby the position within the 
sensor field closest to the part of the user is determined without any 
calibration of the sensor field to the user. 



(Fig. 6) 
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Fig. 3 




Fig. 4 
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